A comprehensive plan of evaluation has been devised that includes assessment measures at numerous points and from numerous perspectives. Evaluation shall be primarily formative, with both qualitative and quantitative data gathered and analyzed as appropriate to evaluate progress on the project goals and objectives.
The scheduled evaluation activities are in the table below.
Evaluation Activity
Product
Year 1
2008-2009
Monitor progress and completion of planning activities related to promotion and startup.
Report on outcomes and effectiveness of initial planning and promotional activities.
Develop instrumentation and survey attendees from first technology club, Guitar Workshop and other high school and middle school activities
Measurement and assessment of changes in awareness, perception, and intention of attendees. Recommendations for changes or modification.
Develop instrumentation and survey attendees from first special event for women.
Measurement and assessment of changes in awareness, perception, and intention of attendees. Recommendations for changes or modification.
Year 2
2009-2010
Monitor organizational effectiveness and progress toward goals and objectives
Qualitative review and report of effectiveness
Survey attendees from Taste of Engineering Careers course, guitar workshop and other events at targeted schools
Measurement and assessment of changes in awareness, perception, and intention of attendees. Recommendations for changes or modification
Monitor development and dissemination of promotional DVD
Survey, measure and report effectiveness of promotional DVD on target audiences.
Create cohort of students to begin follow-up to determine effect of project activities on enrollment decisions.
Gather data and report on extent to which recruitment and promotional activities result in enrollment.
Develop student profile describing relative effectiveness of techniques and approaches.
Correlate second year results with first year results and develop student attitude and effectiveness profile report.
Year 3
2010-2011
Monitor, measure and evaluate overall effectiveness of technology clubs, Taste of Engineering Careers course, guitar workshops, team events and related activities on target groups at target schools
Qualitative and quantitative comparison of expected and actual results. Longitudinal analysis of changes in awareness, perception, and intention of attendees.
Monitor and measure long term effect of recruitment and promotional videos and related materials
Survey, measure and report on effectiveness of materials on target audiences.
Monitor and measure effect of special efforts targeting adults.
Survey, measure and report on effectiveness of materials on adults.
Conduct a project evaluation with Senior Personnel, internal and external evaluators.
Final report
The evaluations and reports in the table above will be applied to the eight project activities:
Special events for high school and middle school students
Inventor’s camps for middle school children and girls
Engineering technology clubs at high schools
A Taste of Engineering Technology course
Leadership teams for high school students
Special high school projects
Developing and dissemination information and promotional materials
Publicizing success
Data developed over the life of this project should show which of these activities are more effective in producing positive changes and generating greater enrollment.
To make the evaluation most effective, survey data from participants will be differentiated according to the following subsets:
Parents
Middle school youth (grade 7-8) and gender (M-F)
High school youth (grade 9-10) and gender (M-F)
High school youth (grade 11-12) and gender (M-F)
Adults (age 22 and + and gender (M-F)
Instrumentation will be developed to discriminate among the following behaviors and intentions:
Behaviors / Intentions
Indicator of Measure
Awareness
Greater knowledge of the career field and general and local/regional opportunities available at varying levels of education.
Perception
Ability to see oneself doing the work. Changes in personal attitude regarding difficulty of math or science oriented careers and subject’s personal ability to succeed.
Intention
Based on information gained from the activity, whether the subject would be more likely to consider enrolling in engineering technology.
Action
Conversion of interest and intention to actual enrollment.
Consideration in the Treatment of Human Subjects
In all cases, data will be gathered with appropriate regard for human subjects and full confidentiality will be maintained. Only aggregated data will be reported. All surveys will be anonymous. Student subjects will have the right to refuse to report any, all, or part of a survey without identification or penalty. Data will only be gathered in common settings and on normal educational practices. No student shall be at risk (as defined by the National Research Act) as a result of data gathering. This research and any data reported there from are not subject to special HHS research guidelines for human subjects and comply with applicable provisions of FERPA.
FOR MORE INFORMATION ABOUT PROJECT ACTIVITIES, contact:
Principal Investigator Dorene Perez, CAD/CAE Program Coordinator, Email: dorene_perez@ivcc.edu
Co-Principal Investigator Jim Gibson, Electronics Program Coordinator, Email: jim_gibson@ivcc.edu
Co-Principal Investigator Sue Caley Opsal, Anatomy and Physiology Instructor, Email: sue_caley@ivcc.edu
Co-Principal Investigator Rose Marie Lynch, Communications Instructor, Email: rosemarie_lynch@ivcc.edu
Illinois Valley Community College
815 North Orlando Smith Avenue
Oglesby, IL 61348
PHONE: 815-224-2720
FAX: 815-224-3033